Doppler-based segmentation and optical flow in radar images

ABSTRACT

In an embodiment, a method for processing an image is provided. The method receives an image including a plurality of pixels. Each pixel includes radial velocity information. The method categorizes the plurality of pixels of the image into a plurality of groups of pixels based on radial velocity information of the pixels. The method associates at least one of the groups of pixels with an object.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a Non-Provisional of U.S. Provisional PatentApplication Ser. No. 62/052,696 filed Sep. 19, 2014, the disclosure ofwhich is incorporated by reference herein in its entirety.

FIELD OF THE INVENTION

The subject invention relates to processing digital images usingcomputer vision techniques and, more specifically, using computer visiontechniques that are modified to use radial velocity information of thepixels of the images when processing the images.

BACKGROUND

Many computer vision techniques for processing digital images have beenused, in order to extract useful information from the digital images.For instance, image segmentation techniques partition an image intomultiple segments in order to locate objects and boundaries (e.g., linesand curves) in images. Optical flow techniques are used to study motionof objects, surfaces, and edges in images caused by the relative motionbetween an observer (an eye or a camera) and the scene. Specifically,optical flow techniques are used to estimate optical flow (i.e.,velocity of movement of a pixel of an image to another location inanother image). Optical flow techniques are used for motion estimation,data compression, robot navigation, and object tracking.

SUMMARY OF THE INVENTION

In one exemplary embodiment of the invention, a method for processing animage is provided. The method receives an image including a plurality ofpixels. Each pixel includes radial velocity information. The methodcategorizes the plurality of pixels of the image into a plurality ofgroups of pixels based on radial velocity information of the pixels. Themethod associates at least one of the groups of pixels with an object.

In another exemplary embodiment of the invention, a system forprocessing an image is provided. The system includes a radar configuredto generate an image that includes a plurality of pixels. Each pixelincludes radial velocity information. The system also includes an imageprocessing module configured to categorize the plurality of pixels ofthe image into a plurality of groups of pixels based on radial velocityinformation of the pixels and associate at least one of the groups ofpixels with an object.

In another exemplary embodiment of the invention, a computer programproduct for processing an image is provided. The computer programproduct comprises a computer readable storage medium having programinstructions embodied therewith. The program instructions are readableby a processing circuit to cause the processing circuit to perform amethod. The method receives an image including a plurality of pixels.Each pixel includes radial velocity information. The method categorizesthe plurality of pixels of the image into a plurality of groups ofpixels based on radial velocity information of the pixels. The methodassociates at least one of the groups of pixels with an object.

The above features and advantages and other features and advantages ofthe invention are readily apparent from the following detaileddescription of the invention when taken in connection with theaccompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

Other features, advantages and details appear, by way of example only,in the following detailed description of embodiments, the detaileddescription referring to the drawings in which:

FIG. 1 depicts a radar and an image processing system in accordance withembodiments of the invention;

FIG. 2 is a flowchart illustrating a method for processing an image inaccordance with embodiments of the invention; and

FIG. 3 is a flowchart illustrating a method for processing a sequence ofimages in accordance with embodiments of the invention.

DESCRIPTION OF THE EMBODIMENTS

The following description is merely exemplary in nature and is notintended to limit the present disclosure, its application or uses. Itshould be understood that throughout the drawings, correspondingreference numerals indicate like or corresponding parts and features.

Images generated by high-resolution imaging radars carry moreinformation than the images generated by conventional radar systems orthe images generated by conventional camera systems. This is because anobject shown in an image generated by a high-resolution imaging radar isrepresented by a larger number of pixels showing more details of theobject while an object shown in an image generated by a conventionalradar system is a lower resolution image where a smaller number ofpixels (e.g., a dot) represent the object. Moreover, an image generatedby a high-resolution imaging radar carries more information than animage generated by a conventional imaging system because a conventionalcamera system does not capture radial velocity information for thepixels in the image.

In embodiments, an image generated by a high-resolution imaging radarincludes radial velocity information in addition to range information,elevation information, and azimuth information, which an image generatedby a conventional radar includes. Specifically, in embodiments, everypixel in an image generated by a high-resolution image radar includesradial velocity information, range information, elevation information,azimuth information and the measured reflected intensities. The radialvelocity information for a pixel indicates a radial velocity of thepixel measured by the radar based on a motion of an object relative tothe radar. The range information for a pixel indicates a distancebetween the object and the radar. The elevation information indicates anelevation angle from the radar to the object. The azimuth informationindicates an azimuth angle from the radar to the object. The radialvelocity information, the range information, the elevation informationand the azimuth information for a pixel are generated by the radar usinga spherical coordinate system. The radar, however, may be configured touse other coordinate system such as a Cartesian coordinate system andgenerate images, of which pixels have coordinate information accordingto the other coordinate system.

Generally speaking, the methods and systems of the embodiments of theinvention adapt computer vision techniques to using the radial velocityinformation for pixels in the images or frames generated by ahigh-resolution image radar and use the adapted computer visiontechniques to process one or more of the images. The computer visiontechniques that the methods and systems of the embodiments of theinvention adapt to using the radial velocity information includesegmentation techniques and optical flow techniques.

In accordance with an exemplary embodiment of the invention, FIG. 1depicts an image processing system 100 and a radar 102. In embodiments,the image processing system 100 includes one or more modules such as adatastore 104, a segmentation module 106 and an optical flow estimationmodule 108. As used herein, the term “module” or “sub-module” refers toan application specific integrated circuit (ASIC), an electroniccircuit, a processor (shared, dedicated, or group) and memory thatexecutes one or more software or firmware programs, a combinationallogic circuit, and/or other suitable components that provide thedescribed functionality. When implemented in software, a module or asub-module can be embodied in memory as a non-transitorymachine-readable storage medium readable by a processing circuit andstoring instructions for execution by the processing circuit forperforming a method. Moreover, the modules shown in FIG. 1 may becombined and/or further partitioned. In embodiments, the imageprocessing system 100 and the radar 102 operate in a vehicle.

The radar 102 generates high-resolution images or frames of theenvironment around the image processing system 100. The radar 102captures sequential images 114 of radar data, which are passed to theimage processing system 100 for image processing and analysis. Inembodiments, the radar 102 is configured to perform a Doppler analysison one or more moving objects 150 that may be present in the range ofthe radar 102 to generate radial velocity information for each pixel ofan image that the radar 102 captures. The radar 102 also generates rangeinformation, elevation information, and azimuth information for eachpixel of the image, in addition to the radial velocity information.Moreover, the radar 102 generates intensity information for each pixelthat indicates one or more intensity or brightness values of the pixel.

The datastore 104 of the image processing system 100 stores the images114 received from the radar 102. In embodiments, an encoder module (notshown) may encode the images 114 before storing the images in thedatastore 104, and a decoder module may decode the encoded images whenthe segmentation module 106 and the optical flow estimation module 108retrieve the images from the datastore 104.

The segmentation module 106 implements one or more segmentationtechniques to apply to the images from the radar 102, in order toidentify objects and boundaries in the images. In embodiments, thesegmentation techniques that the segmentation module 106 implements usethe radial velocity information of the pixels in an image, in additionto other information associated with the pixels such as the intensityinformation. The segmentation module 106 sends the processed images withidentified objects and boundaries to another system (not shown) thatuses the processed images. For instance, the image processing system 100of the embodiments that operates in a vehicle sends the processed imagesto another system (not shown) that implements safety features (e.g.,detecting other vehicles, pedestrians, other obstacles, etc.) for thevehicle.

The segmentation techniques that the segmentation module 106 implementsinclude conventional segmentation techniques that are adapted to usingthe radial velocity information in accordance with the embodiments ofthe invention. There are numerous conventional segmentation techniquesthat may be adapted to using the radial velocity information. Exampleconventional segmentation techniques include region-growing techniques,clustering techniques, edge-based (or edge detection) techniques andmodel-based techniques.

The region-growing techniques start from one or more seed pixels in animage as an object's representative pixels. The region-growingtechniques apply one or more similarity criteria to the pixels thatneighbor the seed pixels in order to determine whether the neighboringpixels are sufficiently similar to the seed pixels to represent the sameobject. In embodiments, the similarity criteria includes whether adifference between the radial velocities of two pixels is smaller than athreshold radial velocity difference. If the difference between theradial velocities is smaller than the threshold difference, theregion-growing techniques determine that the two pixels are sufficientlysimilar to represent the same object. Other similar criteria (e.g.,whether a difference between the intensity values of the two pixels issmaller than a threshold intensity difference), may complement indetermining whether two pixels are sufficiently similar to represent thesame object. The region-growing techniques that are adapted to using theradial velocity information may be used to detect objects in the imagebased on pulse-coupled neural network (PCNN).

The clustering techniques cluster pixels of an image into one or moreregions within the image that represent the same objects based on, e.g.,the similarities in intensity values. In embodiments, the clusteringtechniques are adapted to using the radial velocity information of thepixels in an image. The clustering techniques that the embodiments ofthe invention adapt to using the radial velocity information includeK-means algorithm, which is an iterative technique that is used topartition an image into K clusters, where K is a positive integer. TheK-means algorithm (1) picks K cluster centers, either randomly or basedon heuristic, (2) assigns each pixel in the image to the cluster thatminimizes the “distance” between the pixel and the corresponding clustercenter, (3) re-computes the cluster centers by averaging properties(e.g., intensities) of all pixels in the cluster, and (4) repeats (2)and (3) until convergence is attained (i.e., no pixels change clustersany further). The “distance” between the pixel and the cluster center isthe squared or absolute difference between the properties (e.g.,intensities, color values, locations, etc.), or a weighted combinationof the properties. K can be selected manually, randomly, or by aheuristic. In embodiments, the K-means algorithm is adapted to using theradial velocity information of the pixels such that the “distance”between the pixel and the cluster center includes a squared or absolutedifference between radial velocity of the pixel and the radial velocityof the cluster center.

The edge-based techniques detect an object in an image by identifyingdiscrete edges in the image and connecting the edges into boundaries ofthe objects in the image. The edges occur where, for example, intensityvalues of adjacent pixels change abruptly. In embodiments, theedge-based techniques are adapted to using the radial velocityinformation for pixels in an image to detect edges in the image. Thatis, the adapted edge-based techniques consider abrupt changes in theradial velocities (e.g., a change or a difference between radialvelocities of two pixels is greater than a threshold radial velocitydifference) as an indication of an edge in the image.

The model-based techniques use a probabilistic model (e.g. Markov randomfield) to identify objects in an image. That is, the model-basedtechniques define a probability of a pixel in the image belonging to aparticular object in the image based on the probabilistic models of theobjects. In embodiments, the model-based techniques are adapted to usingthe radial velocity information for the pixels in an image to build aprobabilistic model of an object and to define a probability of a pixelbelonging to the object.

The optical flow estimation module 108 implements one or more opticalflow techniques to estimate optical flow of each pixel of an image in asequence of images generated by the radar 102. In embodiments, theoptical flow techniques that the optical flow estimation module 108implements are adapted to using the radial velocity information of thepixels in an image, in addition to other information associated with thepixels such as the intensity information. The optical flow estimationmodule 108 sends the processed images with the estimated optical flowsto another system (not shown) that uses the processed images. Forinstance, the image processing system 100 of the embodiments thatoperates in a vehicle sends the processed images to another system (notshown) that implements safety features (e.g., responding to motions ofother vehicles, pedestrians, other obstacles, etc.) for the vehicle.

There are numerous conventional optical flow techniques that may beadapted by the embodiments of the invention to using the radial velocityinformation. Example conventional optical flow techniques include LukasKanade (LK) techniques, weighted LK techniques, discrete optimizationtechniques, shift flow techniques and weighted LK techniques withincorporated motion models.

The conventional optical flow techniques are based on estimation ofinstantaneous image velocities or discrete image displacements using asequence of ordered images. The instantaneous image velocities in theCartesian coordinate system are defined by the following Equation 1:

V(r)=[V _(x)(r),V _(y)(r),V _(z)(r)]  Equation 1

where V(r) represents an instantaneous velocity of pixel r from oneimage to the next image in the sequence; r represents pixel coordinatevalues (e.g., x, y, and z coordinate values in Cartesian coordinatesystem) of each particular pixel of an image in the sequence; V_(x)(r)represents x-coordinate pixel velocity of the particular pixel; V_(y)(r)represents y-coordinate pixel velocity of the particular pixel; andV_(z)(r) represents z-coordinate pixel velocity of the particular pixel.The conventional optical flow techniques determine an image velocity foran image by imposing an intensity constancy constraint on Equation 1.The following Equation 2 represents the intensity constancy constraint,which is that the intensities or brightness of pixels in the first imageof two consecutive images in the sequence and the corresponding pixelsin the second image of the two consecutive images do not change:

I(r+V,t+Δt)=I(r,t)  Equation 2

where I(r, t) represents the intensity values of the first image of thetwo images—the intensity at time t and position r (i.e., x, y and zcoordinate values); I(r+V, t+Δt) represents the intensity values of thesecond image of the two images; V represents coordinate vector having x,y and z component vectors; and Δt represents a time gap between twoinstances in time at which the two images were captured. Linearizationof Equation 2 in the Taylor series leads to an approximation representedby the following Equation 3, which is referred to as the intensityconstancy constraint equation:

∇I(r,t)·V(r,t)+I _(t)(r,t)=0  Equation 3

where ∇I is:

$\left\lbrack {\frac{\partial I}{\partial x},\frac{\partial I}{\partial y},\frac{\partial I}{\partial z}} \right\rbrack$

In Equation 3, ∇I(r, t) is a gradient vector of the intensities of thepixels of the image (i.e., I(r)), and I_(t)(r, t) is a partialderivative over time t. (Hereinafter, time t is omitted in the equationsand terms). These values are calculated from consecutive images based onnumerical discretization. The intensity constancy constraint alone,however, is insufficient in solving for the three unknown velocitycomponents (i.e., V_(x), V_(y) and V_(z)). Therefore, in order to solvefor these unknowns, additional constraints are necessary.

Different conventional optical flow techniques are different in thatthey impose different additional constraints. For instance, the LKtechniques impose a local velocity constancy constraint, which is thatV, the coordinate vector having V_(x), V_(y) and V_(z), is assumed to beconstant within a spatial neighborhood of a particular pixel at r (i.e.,a spatial window around a particular pixel at r). That is, Equation 3 isassumed to hold for all neighboring pixels of the particular pixel.Therefore, Equation 3 may be rewritten into Equation 4:

∇I(q)·V(r)+I _(t)(q)=0  Equation 4

where q represents each of the pixels that are spatial neighbors of theparticular pixel r in an image in the sequence; and I_(t)(q) is apartial derivative over time t. The LK techniques then obtain a solutionbased on the least squares principle—the overall solution minimizes thesum of the squares of the errors made in the results of every singleequation. The solution is expressed as Equation 5:

V=(A ^(T) A)⁻¹ A ^(T) b  Equation 5

where A is a n-by-three matrix in which n is the number of neighboringpixels of the particular pixel r and is built by concatenation ofgradients ∇I(q) for neighboring pixels q. That is, the Matrix A in thissolution is (referred to as Matrix 1):

$\begin{matrix}\begin{bmatrix}{\nabla{I_{x}\left( q_{1} \right)}} & {\nabla{I_{x}\left( q_{2} \right)}} & \ldots & {\nabla{I_{x}\left( q_{n} \right)}} \\{\nabla{I_{y}\left( q_{1} \right)}} & {\nabla{I_{y}\left( q_{2} \right)}} & \ldots & {\nabla{I_{y}\left( q_{n} \right)}} \\{\nabla{I_{z}\left( q_{1} \right)}} & {\nabla{I_{z}\left( q_{2} \right)}} & \ldots & {\nabla{I_{z}\left( q_{n} \right)}}\end{bmatrix}^{T} & {{Matrix}\mspace{14mu} 1}\end{matrix}$

The Vector b is given by (referred to as Vector 1):

−[I _(t)(q ₁)I _(t)(q ₂) . . . I _(t)(q _(n))]^(T)  Vector 1

In embodiments, the conventional LK techniques are adapted to using theradial velocity information for each pixel in an image as an additionalconstraint to solve for the image velocity V(r) in Equation 3.Specifically, the radial velocity information provides the radialcomponent of the image velocity V(r). The radial velocity informationfor each pixel as an additional constraint may be expressed as Equation6:

e _(r) ·V(r)=V _(D)(r)  Equation 6

where V(r) is the image velocity of an image in the sequence; V_(D)(r)is the radial velocity of a particular pixel at r; and e_(r) isexpressed as:

$\frac{r}{r}$

This additional linear constraint imposed on Equation 3 facilitates moreaccurate and stable estimation of the optical flow per each pixel. Withthis additional constraint imposed on Equation 3, Matrix A in a solutiongiven by Equation 5 is a 2n-by-three matrix:

$\begin{matrix}\begin{bmatrix}{\nabla{I_{x}\left( q_{1} \right)}} & {\nabla{I_{x}\left( q_{2} \right)}} & \ldots & {\nabla{I_{x}\left( q_{n} \right)}} & e_{x,1} & e_{x,2} & \ldots & e_{x,n} \\{\nabla{I_{y}\left( q_{1} \right)}} & {\nabla{I_{y}\left( q_{2} \right)}} & \ldots & {\nabla{I_{y}\left( q_{n} \right)}} & e_{y,1} & e_{y,2} & \ldots & e_{y,n} \\{\nabla{I_{z}\left( q_{1} \right)}} & {\nabla{I_{z}\left( q_{2} \right)}} & \ldots & {\nabla{I_{z}\left( q_{n} \right)}} & e_{z,1} & e_{z,2} & \ldots & e_{{z,n}\;}\end{bmatrix}^{T} & {{Matrix}\mspace{14mu} 2}\end{matrix}$

where e_(r,n) is expressed as:

$\frac{q_{n}}{q_{n}}$

Then, the Vector b is given by (referred to as Vector 2):

−[I _(t)(q ₁)I _(t)(q ₂) . . . I _(t)(q _(n))−V _(D)(q ₁)−V _(D)(q ₂) .. . −V _(D)(q _(n))]^(T)  Vector 2

Doubling the number of constraints allows for using a smaller number ofneighboring pixels for each particular pixel (i.e., a reduction of thewindow size by √{square root over (2)} for two-dimensional radar imagesor by √{square root over (2)} for three-dimensional radar images). Also,using the radial velocity information as a constraint results in abetter estimation of optical flow than using the local velocityconstraint imposed by the conventional LK techniques.

The least squares principle provides the same importance to allneighboring pixels for a particular pixel. In the weighted LKtechniques, the neighboring pixels are weighted. Therefore, inembodiments, Matrix 1 is re-written as Matrix 3 to include weights:

$\begin{matrix}\begin{bmatrix}{w_{1}{\nabla{I_{x}\left( q_{1} \right)}}} & {w_{2}{\nabla{I_{x}\left( q_{2} \right)}}} & \ldots & {w_{n}{\nabla{I_{x}\left( q_{n} \right)}}} \\{w_{1}{\nabla{I_{y}\left( q_{1} \right)}}} & {w_{2}{\nabla{I_{y}\left( q_{2} \right)}}} & \ldots & {w_{n}{\nabla{I_{y}\left( q_{n} \right)}}} \\{w_{1}{\nabla{I_{z}\left( q_{1} \right)}}} & {w_{2}{\nabla{I_{z}\left( q_{2} \right)}}} & \ldots & {w_{n}{\nabla{I_{z}\left( q_{n} \right)}}}\end{bmatrix}^{T} & {{Matrix}\mspace{14mu} 3}\end{matrix}$

Moreover, Vector 1 may be re-written as Vector 3:

−[w ₁ I _(t)(q ₁)w ₂ I _(t)(q ₂) . . . w _(n) I _(t)(q_(n))]^(T)  Vector 3

It is to be noted that, in embodiments, Matrix 2 is similarly modifiedto include the weights. Matrix 2 re-written to include the weights isthe following Matrix 4:

$\begin{matrix}\begin{bmatrix}{w_{1}{\nabla{I_{x}\left( q_{1} \right)}}} & {w_{2}{\nabla{I_{x}\left( q_{2} \right)}}} & \ldots & {w_{n}{\nabla{I_{x}\left( q_{n} \right)}}} & {w_{1}e_{x,1}} & {w_{2}e_{x,2}} & \ldots & {w_{n}e_{x,n}} \\{w_{1}{\nabla{I_{y}\left( q_{1} \right)}}} & {w_{2}{\nabla{I_{y}\left( q_{2} \right)}}} & \ldots & {w_{n}{\nabla{I_{y}\left( q_{n} \right)}}} & {w_{1}e_{y,1}} & {w_{2}e_{y,2}} & \ldots & {w_{n}e_{y,n}} \\{w_{1}{\nabla{I_{z}\left( q_{1} \right)}}} & {w_{2}{\nabla{I_{z}\left( q_{2} \right)}}} & \ldots & {w_{n}{\nabla{I_{z}\left( q_{n} \right)}}} & {w_{1}e_{z,1}} & {w_{2}e_{z,2}} & \ldots & {w_{n}e_{{z,n}\;}}\end{bmatrix}^{T} & {{Matrix}\mspace{14mu} 4}\end{matrix}$

Vector 3 is re-written as (referred to as Vector 4):

[w ₁ I _(t)(q ₁)w ₂ I _(t)(q ₂) . . . w _(n) I _(t)(q _(n))−w ₁ V _(D)(q₁)−w ₂ V _(D)(q ₂) . . . −w _(n) I _(t) V _(D)(q _(n))]   Vector 4

In embodiments, the weights in Matrices 3 and 4 and Vectors 3 and 4 aremodeled as a kernel function and decrease with the distance between theneighboring pixels and the particular pixel at r. The weights modeled asa kernel function may be expressed as Equation 7:

w _(k) =K _(s)(∥q _(k) −r∥)  Equation 7

where K_(s) is a kernel function.

Another way of using the radial velocity information of the pixels as aconstraint is transferring the local velocity constancy constraint ofthe LK techniques to corresponding relationship for radial velocityinformation and using the corresponding relationship for modeling theweighting coefficients w_(k). That is, the weighting coefficients may bedetermined based on an assumption that the radial velocities ofneighboring pixels of a particular pixel are constant. This assumptionleads to the following Equations 8 and 9:

V _(D)(q _(k))=∥V(r)∥cos(e _(k) ,V(r))  Equation 8

V _(D)(r)=∥V(r)∥cos(e _(r) ,V(r))  Equation 9

where V_(D) (q_(k)) represents the radial velocity of a neighboringpixel at q_(k); and V_(D)(r) is the radial velocity of a particularpixel at r. A difference between radial components is computed bysubtracting Equation 9 from Equation 8 as the following Equation 10shows:

$\begin{matrix}\begin{matrix}{{{{V_{D}\left( q_{k} \right)} - {V_{D}(r)}}} = {{{V(r)}} \cdot {{{\cos \left( \left\lbrack {e_{r},{V(r)}} \right\rbrack \right)} - {\cos \left( \left\lbrack {e_{k},{V(r)}} \right\rbrack \right)}}}}} \\{= {2{{{V(r)}} \cdot {{{\sin \left( \frac{\delta_{k}}{2} \right)}{\sin \left( {\left\lbrack {e_{r},{V(r)}} \right\rbrack + \frac{\delta_{k}}{2}} \right)}}}}}} \\{= {2{{{\sin \left( \frac{\delta_{k}}{2} \right)}\left( {{{V_{\tau}(r)}{\cos \left( \frac{\delta_{k}}{2} \right)}} + {{V_{d}(r)}{\sin \left( \frac{\delta_{k}}{2} \right)}}} \right)}}}} \\{= {{{{V_{\tau}(r)}{\sin \left( \delta_{k} \right)}} + {{V_{D}(r)}\left( {1 - {\cos \left( \delta_{k} \right)}} \right)}}}}\end{matrix} & {{Equation}\mspace{14mu} 10}\end{matrix}$

where δ_(k) is an angle between e_(r) and e_(q) _(k) . This assumptionthat δ_(k) does not change considerably within the spatial neighborhoodof a particular pixel r (i.e., δ_(k) is small) allows for modeling theweighting coefficients as the following Equation 11 shows:

w _(k) =K _(s)(∥q _(k) −r∥)·K _(D)(∥V _(D)(q _(k))−V _(D)(r)∥)  Equation11

wherein K_(D) is a kernel function.

In embodiments, the weighting coefficients computed based on Equation 11are used in the solution computed by imposing the local velocityconstancy constraint, i.e., Matrix 1 and Vector 1, as well as in thesolution computed by imposing the radial velocity constraint, i.e.,Matrix 2 and Vector 2. Moreover, in embodiments, the kernel functionK_(D) may be corrected as shown in Equation 12:

$\begin{matrix}{w_{k} = {{K_{s}\left( {{q_{k} - r}} \right)} \cdot {K_{D}\left( \frac{{{V_{D}\left( q_{k} \right)} - {V_{D}(r)}}}{1 - {\cos \left( \delta_{k} \right)}} \right)}}} & {{Equation}\mspace{14mu} 12}\end{matrix}$

It is to be noted that the radial velocity constraints of theembodiments of the invention are applicable to the Horn-Schunckalgorithm for motion estimation that uses smoothness constraint on themotion field and switches to robust methods assuming non-Gaussianmean-zero assumptions:

ε(r)=∇I(r,t)·V(r,t)+I _(t)(r,t)  Equation 13

The Horn-Schunck algorithm is described in B. K. P. Horn and B. G.Schunk, Determining optical flow, Artificial Intelligence, vol.17:185-203, 1981, which is incorporated herein by reference.

The radial velocity constraints of the embodiments of the invention mayalso be used as constraints on the motion field in the probabilisticformulation of the motion estimation described in D. J. Fleet and Y.Weiss, “Optical Flow Estimation”, chapter 5, 2006, which is incorporatedherein by reference. The radial velocity constraints of the embodimentsof the invention may be incorporated into the mixture models addressedusing expectation-maximization (EM) approach, described in A. Jepson andM. J. Black, “Mixture models for optical flow computation”, In Proc.IEEE Computer Vision and Pattern Recognition, CVPR-93, pages 760-761,New York, June 1993, which is incorporated herein by reference.

Referring now to FIG. 2, and with continued reference to FIG. 1, aflowchart illustrates a method for compressing a sequence of images. Inembodiments, the method can be performed by the image processing system100 of FIG. 1. As can be appreciated in light of the disclosure, theorder of operation within the method is not limited to the sequentialexecution as illustrated in FIG. 2, but may be performed in one or morevarying orders as applicable and in accordance with the presentdisclosure. In embodiments, the method can be scheduled to run based onpredetermined events, and/or run continually during operation of theimage processing system 100.

At block 210, the image processing system 100 receives an imageincluding a plurality of pixels from the radar 102. Each of the pixelsincludes radial velocity information. The radial velocity informationindicates a radial velocity of each of the pixels in the plurality ofpixels measured by the radar 102 based on a motion of an object relativeto the radar 102. Each of the pixels also includes intensityinformation.

At block 220, the image processing system 100 categorizes the pluralityof pixels of the image into a plurality of groups of pixels based onradial velocity information of the pixels as well as the intensityinformation of the pixels. The image processing system 100 determineswhether one pixel of the plurality of pixels is sufficiently similar toanother pixel of the plurality of pixels based on the radial velocityinformation for the two pixels, by comparing the radial velocityinformation for the one pixel and the radial velocity information forthe other pixel. In embodiments, the image processing system 100 uses animage segmentation technique that is adapted to using the radialvelocity information of the pixels in order to categorize the pixels.

At block 230, the image processing system 100 associates at least one ofthe groups of pixels with an object.

Referring now to FIG. 3, and with continued reference to FIG. 1, aflowchart illustrates a method for compressing a sequence of images. Inembodiments, the method can be performed by the image processing system100 of FIG. 1. As can be appreciated in light of the disclosure, theorder of operation within the method is not limited to the sequentialexecution as illustrated in FIG. 3, but may be performed in one or morevarying orders as applicable and in accordance with the presentdisclosure. In embodiments, the method can be scheduled to run based onpredetermined events, and/or run continually during operation of theimage processing system 100.

At block 310, the image processing system 100 receives a sequence ofimages from the radar 102. Each image in the sequence includes aplurality of pixels. Each of the pixels includes radial velocityinformation. The radial velocity information indicates a radial velocityof each of the pixels in the plurality of pixels measured by the radar102 based on a motion of an object relative to the radar 102. Each ofthe pixels includes intensity information. Each pixel in the pluralityof pixels includes location information such as an x-coordinate value, ay-coordinate value and a z-coordinate value.

At block 320 the image processing system 100 estimates optical flow forthe pixels within an image in the sequence of images by using the radialvelocity information for the pixels as a constraint. The radial velocityinformation for each of the pixels is used as a constraint to solve theintensity constancy constraint equation for the optical flow for thepixels based on assumption that the radial velocities of a plurality ofpixels that are spatially adjacent to a particular pixel in an image areconstant. The image processing system 100 also uses the intensityinformation as another constraint. The image processing system 100 usesan optical flow technique that is adapted to using the radial velocityinformation of the pixel in order to estimate optical flow for thepixel.

While the invention has been described with reference to exemplaryembodiments, it will be understood by those skilled in the art thatvarious changes may be made and equivalents may be substituted forelements thereof without departing from the scope of the invention. Inaddition, many modifications may be made to adapt a particular situationor material to the teachings of the invention without departing from theessential scope thereof. Therefore, it is intended that the inventionnot be limited to the particular embodiments disclosed, but that theinvention will include all embodiments falling within the scope of theapplication.

What is claimed is:
 1. A computer-implemented method for processing animage, comprising: receiving an image including a plurality of pixels,each pixel including radial velocity information; categorizing, by acomputer, the plurality of pixels of the image into a plurality ofgroups of pixels based on radial velocity information of the pixels; andassociating at least one of the groups of pixels with an object.
 2. Themethod of claim 1, wherein the radial velocity information indicates aradial velocity of each of the pixels measured by a radar based on amotion of an object relative to the radar.
 3. The method of claim 1,further comprising using an image segmentation technique that is adaptedto using the radial velocity information of the pixels.
 4. The method ofclaim 1, wherein each pixel further includes intensity information,wherein the categorizing is further based on the intensity informationof the pixels.
 5. The method of claim 1, wherein the categorizingcomprises determining whether one pixel of the plurality of pixels issufficiently similar to another pixel of the plurality of pixels basedon the radial velocity information for the two pixels.
 6. The method ofclaim 1, wherein the categorizing comprises comparing the radialvelocity information for one pixel of the plurality of pixels and theradial velocity information for another pixel of the plurality ofpixels.
 7. A system for processing an image, comprising: a radarconfigured to generate an image that includes a plurality of pixels,each pixel including radial velocity information; and an imageprocessing module configured to: categorize the plurality of pixels ofthe image into a plurality of groups of pixels based on radial velocityinformation of the pixels; and associate at least one of the groups ofpixels with an object.
 8. The system of claim 7, wherein the radialvelocity information indicates a radial velocity of each of the pixelsmeasured by a radar based on a motion of an object relative to theradar.
 9. The system of claim 7, wherein the image processing module isconfigured to use an image segmentation technique that is adapted tousing the radial velocity information of the pixels.
 10. The system ofclaim 7, wherein each pixel further includes intensity information,wherein the categorizing is further based on the intensity informationof the pixels.
 11. The system of claim 7, wherein the image processingmodule is configured to categorize by determining whether one pixel ofthe plurality of pixels is sufficiently similar to another pixel of theplurality of pixels based on the radial velocity information for the twopixels.
 12. The system of claim 7, wherein the image processing moduleis configured to categorize by comparing the radial velocity informationfor one pixel of the plurality of pixels and the radial velocityinformation for another pixel of the plurality of pixels.
 13. A computerprogram product for processing an image, the computer program productcomprising: a computer readable storage medium having programinstructions embodied therewith, the program instructions readable by aprocessing circuit to cause the processing circuit to perform a methodcomprising: receiving an image including a plurality of pixels, eachpixel including radial velocity information; categorizing the pluralityof pixels of the image into a plurality of groups of pixels based onradial velocity information of the pixels; and associating at least oneof the groups of pixels with an object.
 14. The computer program productof claim 13, wherein the radial velocity information indicates a radialvelocity of the pixel measured by a radar based on a motion of an objectrelative to the radar.
 15. The computer program product of claim 13,wherein the method further comprises using an image segmentationtechnique that is adapted to using the radial velocity information ofthe pixels.
 16. The computer program product of claim 13, wherein eachpixel further includes intensity information, wherein the categorizingis further based on the intensity information of the pixels.
 17. Thecomputer program product of claim 13, wherein the categorizing comprisesdetermining whether one pixel of the plurality of pixels is sufficientlysimilar to another pixel of the plurality of pixels based on the radialvelocity information for the two pixels.
 18. The computer programproduct of claim 13, wherein the categorizing comprises comparing theradial velocity information for one pixel of the plurality of pixels andthe radial velocity information for another pixel of the plurality ofpixels.